1. Field of Invention
The present invention relates to a three-dimensional (3D) human-computer interaction system that supports mouse operations through the motion of a finger and an operation method thereof, in which the system has an information connection with an electronic information device, so that an operator is enabled to express a motion by using a finger and perform operations on an operation interface of the electronic information device in a 3D space in a noncontact manner; and the present invention particularly relates to a 3D human-computer interaction system that supports mouse operations through the motion of a finger and an operation method thereof, in which a multi-view video recording device is used to capture an image of a finger of an operator, perform interpretation and achieve command recognition.
2. Related Art
At present, the most famous physical products of a human-computer interface interaction system include the X-BOX from Microsoft Company and the Wii manufactured by Nintendo Company. In the human-computer interface interaction, an electronic information device senses a motion of an operator and interprets the sensed result into a corresponding motion command. Currently, the technologies using gestures as a computer input device have already been disclosed or applied for patents. Most of the technologies are based on the function of simulating mouse operations by using a two-dimensional (2D) input device, for example:
1. U.S. Pat. No. 6,057,540 “Mouseless optical and position translation type screen pointer control for a computer system”;
2. U.S. Pat. No. 7,274,800 “Dynamic gesture recognition from stereo sequences”; and
3. U.S. Pat. No. 7,849,421 “Virtual mouse driving apparatus and method using two-handed gestures”.
The above patents are the disclosure of the prior art of the human-computer interaction, which can be roughly classified into the following categories:
(1) Using combinations of the left and right gestures to form various mouse commands, and the image capture device is an ordinary single-view webcam;
(2) Using 2D motions of the finger and some additional keys to replicate mouse commands, and the image capture device consists of a single-view webcam and a movement sensor;
(3) Using motions of a hand or arms, including depth information, to define various mouse commands, and the image capture device consists of multiple sets of cameras; and
(4) Using 2D/3D motions of a hand or a finger to define various mouse commands, and the image capture device consists of an infrared laser and an infrared camera, or a set of infrared diodes and a set of infrared sensors.
The apparatuses/techniques in categories (1) and (2) are mainly applied to ordinary digital cameras to capture a motion image and then interpret a motion command. The acquired image information is limited to 2D information. The 3D motion part further requires at least one movement sensor for combination.
As far as the apparatus/technique in category (3) is concerned, a 3D camera or multiple sets of cameras are used to capture an image. The apparatuses have a large number of components and are complicated. Moreover, the whole implementation result is affected by whether the cameras may be precisely synchronized. Also, after the images captured by a numbers of image capturing devices, depth computation needs to be performed on each whole image to interpret the image into a motion command. Once a small problem occurs on the definition of the parameter, distortion may be caused on the interpretation result.
As far as the apparatus/technique in category (4) is concerned, more different cameras and sensors need to be combined. The integral formation complexity is much higher than the apparatuses/techniques in categories (1) and (2).